Systems for spectral multiplexing of source images to provide a composite image with gray component replacement, for rendering the composite image, and for spectral demultiplexing of the composite image

ABSTRACT

Methods and apparatus for spectrally-encoding plural source images and for providing the spectrally-encoded plural source images in a composite image, for rendering the composite image in a physical form, or for recovering at least one of the encoded source images from the rendered composite image such that the recovered source image is made distinguishable. Source image confusion in a rendered composite image is controlled by application of a multi-illuminant gray component replacement (GCR) technique to the darkness common to the different colorants under the multiple illuminants.

CROSS-REFERENCE TO RELATED APPLICATIONS

Cross-reference is made to the following contemporaneously filed,co-pending, and commonly-owned applications:

U.S. application Ser. No. 10/268,333, filed Oct. 9, 2002 by GauravSharma et al. and respectively entitled “Systems for spectralmultiplexing of source images to provide a composite image, forrendering the composite image, and for spectral demultiplexing of thecomposite image”.

U.S. application Ser. No. 10/10/268,394 filed Oct. 9, 2002 by Steven J.Harrington et al. and respectively entitled “Systems for spectralmultiplexing of source images to provide a composite image, forrendering the composite image, and for spectral demultiplexing of thecomposite image to animate recovered source images”.

U.S. application Ser. No. 10/268,295 filed Oct. 9, 2002 by Steven J.Harrington et al. and respectively entitled “Systems for spectralmultiplexing of source images including a stereogram source image toprovide a composite image, for rendering the composite image, and forspectral demultiplexing of the composite image”.

U.S. application Ser. No. 10/268,505 filed Oct. 9, 2002 by Steven J.Harrington et al. and respectively entitled “Systems for spectralmultiplexing of source images including a textured source image toprovide a composite image, for rendering the composite image, and forspectral demultiplexing of the composite image”.

U.S. application Ser. No. 10/268,246 filed Oct. 9, 2002 by Robert P.Loce et al. and respectively entitled “Systems for spectral multiplexingof a source image and a background image to provide a composite image,for rendering the composite image, and for spectral demultiplexing ofthe composite images”.

U.S. application Ser. No. 10/268,271 filed Oct. 9, 2002 by Yeqing Zhanget al. and respectively entitled “Systems for spectral multiplexing ofsource images to provide a composite image, for rendering the compositeimage, and for spectral demultiplexing the composite image, whichachieve increased dynamic range in a recovered source image”.

U.S. application Ser. No. 10/268,241 filed Oct. 9, 2002 by Yeqing Zhanget al. and respectively entitled “System for spectral multiplexing ofsource images to provide a composite image with noise encoding toincrease image confusion in the composite image, for rendering thecomposite image, and for spectral demultiplexing of the compositeimage”.

U.S. application Ser. No. 10/304,171 filed Nov. 25, 2002 by GauravSharma et al. and respectively entitled “Systems for spectralmultiplexing of source images to provide a composite image, forrendering the composite image, and for spectral demultiplexing of thecomposite image by use of an image capture device”.

FIELD OF THE INVENTION

The present invention relates to a system or systems for spectrallymultiplexing a plurality of source images so as to provide a compositeimage, rendering the composite image, and demultiplexing of such acomposite image to recover one or more of the source images.

BACKGROUND OF THE INVENTION

Whereas light sources are visible by their own emitted light, objectsand materials appear to the eye according to how they modify incidentlight. The sensation of the color of an object is evoked by the physicalstimulation of light-sensitive receptors in the human retina. Thestimulation consists of electromagnetic radiation in the visiblespectrum comprising wavelengths between about 380 and 780 nanometers.

Perceived color of the object is the result of a combination of factors,such as: (1) the spectral power distribution of an illuminant emitted bya light source that is incident upon the object, (2) the modification ofthe spectral power distribution of the illuminant by the spectralreflectance or transmission characteristics of the illuminated object,(3) the excitation of light sensitive receptors in the eye by themodified light from the object, and (4) the perception andinterpretation by the brain of signals produced by the light sensitivereceptors.

The perception of color is attributed to the differing spectralsensitivities of the light sensitive receptors. The trichromacy of colorsensation implies that many different spectral distributions can producethe same perceived color. Such equivalent stimuli, which produce thesame perception even though they are physically different spectraldistributions, are called metamers, and the phenomena metamerism. Forexample, it is known that the perceived color of an object can changequite markedly when the object is moved from incident daylight intoincident artificial light. The spectrum of the illuminating light sourceis also known to have an effect on the perceived colors of a printedimage in spite of the considerable physiological compensation that theeye makes for differences in illumination. Light sources of differingrelative spectral power distributions are therefore known to havedifferent color rendering properties: for example, light sources whichemit very narrow band, or almost monochromatic, light are considered torender colors very poorly.

According to the concept of metamerism, the respective colors of twoobjects may appear to be identical even though typically the spectralpower distributions produced from the objects are different. Such powerdistributions, or stimuli, which are spectrally different but visuallyidentical, are considered as metameric pairs. Because we measure lightusing only three cone types, the differences in these powerdistributions are indistinguishable. Two objects with different spectralreflectance functions may be perceived to match in color under oneilluminant and not match under a different illuminant.

Certain aspects of perceived color have been employed to disguise imagesby printing an image in one color and then overprinting the first imagewith a pattern in a different color having approximately the sameapparent brightness. Adjacent zones of equal brightness appear to bevisually blended, even though they are of differing colors, therebyconfusing the perception of the original image.

It is known to print patterns in different colors such that the patternsmay be viewed through one or more filters having certain correlatedcolors, such that the patterns will change, depending upon the colorsinvolved. It is also known to print characters in different colors in anoverlapping relationship such that the overlapped characters, whenviewed through one colored filter, will give the appearance of onlycertain ones of the superimposed characters, and when viewed through asecond and differing colored filter, will reveal certain other ones ofthe superimposed characters. Such approaches are known for encoding (orencrypting) information to prevent recognition of the informationcontent of the pattern until the pattern is decoded and madecomprehensible. These approaches have been applied to promotional gamingtechnology and in document security and document verificationapplications.

Techniques are known for rendering flat, two-dimensional images that canstimulate an illusion of depth perception, that is, of athree-dimensional object or scene. Three-dimensional imaging can beclassified into two major groups according to the quantity ofinformation required to record the images: (1) binocular stereoscopicimaging, and (2) autostereoscopy, or three-dimensional spatial imaging.See Takanori Okoshi, Three-dimensional Imaging Techniques, AcademicPress Inc., New York, (1976). Devices for performing binocularstereoscopic imaging include binocular viewers, parallaxstereogramslenticular-sheet binocular stereoscopic pictures, andbinocular displays using Polaroid glasses or color filters. Devices forperforming autostereoscopy include parallax panoramagrams,lenticular-sheet three-dimensional imaging, projection typethree-dimensional displays, and integral photography.

In stereoscopy, a three-dimensional image is created by a series oftwo-dimensional images of an object captured from differentperspectives, and therefore the three-dimensional image so producedcontains multiple-angle information about the object. Physicallydisplaced views of the same image are presented simultaneously to theeyes of an observer to convey the illusion of depth. These techniquestypically employ a multiplexed pair of images, wherein the images arenearly identical and differ only so as to simulate parallax. Themultiplexing is performed according to color, polarization, temporal, orposition differences between the constituent images. For example,anaglyphic stereoscopy is a well-known process, in which left and rightnearly-identical images are color-encoded by use of respectivecomplementary color filters (e.g. cyan and red) for subsequent viewingthrough correspondingly colored lenses to separate the images asnecessary for a simulated three-dimensional effect. When viewed throughcolored spectacles, the images merge to produce a stereoscopicsensation. The encoded image pair is known as an anaglyph, as it istypically rendered as two images of the same object taken from slightlydifferent angles in two complementary colors.

This stereoscopic viewing of the multiplexed image pair typicallyrequires the use of optical devices to channel each of the paired (leftand right) images solely to the appropriate eye of the observer. A fewautostereoscopic display techniques are known for providing subjectivelythree-dimensional viewing of a fixed image plane, without resort toeyewear and the like, by use of alternative devices based upondirection-multiplexed image displays. These devices typically employoptical diffraction, lenticular imaging, or holographic phenomena.

SUMMARY OF THE INVENTION

Spectral multiplexing, as used herein, refers to a process for encodingplural source images in a composite image. Composite image renderingrefers to a process for rendering the composite image in a physicalform. Spectral demultiplexing refers to a process for recovering atleast one of the encoded source images from the rendered compositeimage, such that the recovered source image is made distinguishablefrom, or within, the composite image, by subjecting the renderedcomposite image to a narrow band illuminant that is preselected toreveal the source image.

Accordingly, the present invention is directed to methods and apparatusfor spectrally-encoding plural source images and for providing thespectrally-encoded plural source images in a composite image, forrendering the composite image in a physical form, or for recovering atleast one of the encoded source images from the rendered composite imagesuch that the recovered source image is made distinguishable. That is,when the rendered composite image is subjected to illumination by one ofthe narrow band illuminants for which a source image was encoded, thesource image becomes visually detectable by an observer. An illuminantthat is designed to particularly interact with a given colorant is saidto be complementary, and vice versa.

Each source image is spectrally encoded by mapping values representativeof each source image pixel to a corresponding pixel value in one or moreof a plurality of colorant image planes. The contemplated encoding, inits simplest form, may include the conversion of each source image to amonochromatic, separation image, which is then directly mapped to acorresponding colorant image plane in the composite image. A pluralityof source images can thereby be mapped to a corresponding plurality ofcolorant image planes in the composite image.

The plural monochromatic separations are designed to be combined in thecomposite image, which in turn will control the amount of one or morepreselected colorants to be deposited on the substrate. In one possibleexample, each colorant is assigned to a respective colorant image planeof the composite image, and the colorant values in the respectivecolorant image planes represent the relative amounts of colorantdeposited in the rendered composite image. For example, a renderedcomposite image may be rendered using cyan, magenta and yellow colorantsthat are deposited over a given area on a substrate by a renderingdevice.

A composite image composed of the plural colorant image planes may bestored or transmitted as composite image file. The composite image maythen be physically realized by delivering the composite image file to arendering device with instructions for rendering the composite image ona substrate using the identified colorant or array of colorants. Onesuitable embodiment of a rendering device therefore includes a digitalcolor electrophotographic printer.

In another embodiment of the contemplated encoding, the mapping of eachsource image is instead performed according to determinations describedherein for compensating the effect of one or more of the following onthe composition, rendering, or demultiplexing of the composite image:(a) the trichromacy of human visual response to colorant/illuminantinteraction; (b) the spectral characteristics of the colorants selectedfor rendering the composite image, such spectral characteristicsespecially comprehending the interaction of plural colorants when suchare combined on the substrate, and (c) the spectral characteristics ofthe narrow band illuminant(s) that will be used to illuminate thecomposite image for recovering the source image.

It will no doubt be appreciated that the encoding of source images tothe composite image may be accomplished according to the teachingsherein with use of either software, hardware or combinationsoftware-hardware implementations.

Accordingly, the present invention is directed to a system forspectrally-encoding plural source images and for providing thespectrally-encoded plural source images in a composite image.

The present invention is also directed to a system for rendering aspectrally-multiplexed composite image on a substrate.

The present invention is also directed to a system for spectraldemultiplexing of a source image presented in a composite imagepresented on a substrate.

In a feature of the present invention, the source image presented in arendered composite image is recovered when the composite image isilluminated by a controlled field of illumination of at least one narrowband illuminant having a selected spectral power distribution.

In another feature of the present invention, source image datarepresentative of a plurality of disparate, pictorial source images maybe spectrally encoded to form secondary image data representative of acomposite image. The composite image may be realized as a single,complex, rendered pattern of deposited colorants, wherein at least onecolorant is utilized for its particular spectral reflectancecharacteristic, and in particular for its narrow band absorptioncharacteristic. The source images are accordingly spectrally multiplexedto provide a composite image which is recorded on the substrate by useof at least one of the narrow band-absorbing colorants. Typically, atleast one of the source images is not easily recognized as such in therendered composite image. That is, until the rendered composite image issubjected to the demultiplexing process, the rendered composite imageis, to a certain extent, visually confused such that an observer issubstantially unable to discern one or more of the source images byunaided viewing of the rendered composite image. Alternatively, one ormore of the source images may be encoded so as avoid or reduce visualconfusion and therefore be visually apparent in the rendered compositeimage when the rendered composite image is subjected to ambient whitelight or another wide band illuminant, and become confused or difficultto detect when the rendered composite image is subjected to acomplementary narrow band illuminant.

In another feature of the present invention, a colorant selected for itsnarrow band absorbing properties may be employed to appear dark whensubjected to its complementary narrow band illuminant, and to appearlight when subjected to a differing illuminant having a spectral powerdistribution that lies substantially outside of the spectral absorptionband of the particular colorant. For example, a cyan colorant may beselected for its absorption of red light, and accordingly the regions ofa rendered composite image that are composed of a cyan colorant willexhibit high darkness under red light. The cyan colorant will exhibitlow darkness under blue light, and will exhibit intermediate darknessunder green light. Likewise, a magenta colorant will exhibit highdarkness under green light, low darkness under red light, and anintermediate darkness under blue light. A yellow colorant will exhibithigh darkness under blue light, low darkness under red light, and anintermediate darkness under green light.

In another feature of the present invention, by using cyan, magenta, andyellow colorants and narrow band red, green, blue illuminants, threesource images may be encoded and rendered using each of the respectivecolorants, and each of the corresponding source images aredistinguishable within the rendered composite image when the renderedcomposite image is subjected to illumination by the red, green, and blueilluminants.

An embodiment of the system for spectral multiplexing of plural sourceimages includes a spectral multiplexer for receiving image datarepresentative of a plurality of source images and for processing theimage data to encode the plurality of source images into a compositeimage data signal.

An embodiment of the spectral multiplexer may be provided in the form ofa computer for receiving image data files representative of a pluralityof source images and for encoding the image data files as a compositeimage data file, and a composite image file storage and/or transmissionmeans connected to the computer.

An embodiment of the system for rendering the composite image includesan image recording device which is responsive to the system for spectralmultiplexing for receiving the composite image data file and forrendering the corresponding composite image on a substrate.

An embodiment of the image recording device may be provided in the formof a printer connected to the composite image file storage and/ortransmission means, for printing the composite image on a substrate.

An embodiment of the printer may include colorants in the form of cyan,yellow, and black pigments, inks, or dyes selected for their apparentdarkness when exposed to at least one of red and blue illuminants, orcolorants in the form of cyan, magenta, and black pigments, inks, ordyes selected for their apparent darkness when exposed to at least oneof red and green illuminants.

An embodiment of the system for spectral demultiplexing of a renderedcomposite image may include a demultiplexer for subjecting the renderedcomposite image on a substrate to illumination by a narrow bandilluminant having a selected spectral power distribution, such that atleast one of the encoded source images is made distinguishable.

An embodiment of the demultiplexer may include an illuminant sourceresponsive to manual control, or a controller and an illuminant sourceresponsive to control by illuminant source control signals provided bythe controller. An embodiment of the illuminant source may include oneor more light sources for providing a defined field of red, green, orblue light.

An embodiment of the controller may include a computer, operableaccording to control programs for generating one or more of theilluminant source control signals, and an illuminant source responsiveto the illuminant source control signals for generating a defined fieldof illumination of narrow band illuminant, whereby a rendered compositeimage on a substrate may be located within the field of illumination andthereby subjected to illumination by the narrow band illuminant.

Gray component replacement (GCR) is a technique used in conventionalcolor imaging, wherein a black colorant is used to reduce the amount ofother colorants in printing any image. Conceptually, the technique maybe understood as a partial or complete replacement, by use of a blackcolorant, of the common “gray” component of a combination of colorantssuch as cyan, magenta, and yellow colorants. A description of theconventional GCR technique can be found, for instance, in Yule,Principles of Color Reproduction, John Wiley and Sons Inc, (1967).

A rendered composite image, when illuminated by white light, mayunintentionally reveal one or more of the source images encoded therein,or, one encoded source image may be more detectable than the otherencoded source images in the presence of white light. Accordingly, theconfusion between the encoded source images may be less than desirableand a more dominant source image may become discernible. This decreasein confusion may especially occur when source images are encoded and thecomposite image is rendered in cyan and yellow colorants; the imagewisepattern of the cyan colorant can dominate the appearance of theimagewise pattern of the yellow colorant when both are subjected towhite light.

Hence, in certain applications of the present invention, it may bedesired that the rendered composite image under white light shall appearsufficiently confused so that none of the encoded source images arevisually discernible.

Alternatively, in certain other applications of the present invention,it may be desired that the rendered composite image under white lightshall appear less confused, such that a dominant one of the encodedsource images is intended to be visually discernible.

Accordingly, an embodiment of the present invention may be provided in amethod for controlling source image confusion in a rendered compositeimage by application of a multi-illuminant gray component replacementtechnique to the darkness common to the different colorants under themultiple illuminants. This method is generally described herein as anaddition of a multiple-illuminant GCR technique to the above-describedmethodology for the composition and rendering of a composite image.

Implementation of this multiple-illuminant GCR technique to thecomposition and rendering of a composite image will provide a more orless confused appearance, depending upon certain factors, in therendered composite image when illuminated by white light. For example,black can be used to replace a portion of the common density of cyan forrecovery of a source image under red light, and black can be used toreplace a portion of the common density of yellow for recovery of asource image under blue light. Common image density produced with thisblack component is more perceptible under white light than the sameimage features produced with only cyan and yellow.

In another embodiment of the present invention, a multiple-illuminantGCR technique may be employed in the encoding and rendering of acomposite image, wherein a particular source image may be encoded forthe purpose of being discernible under white light.

In another embodiment of the present invention, a gray componentreplacement fraction in a multiple-illuminant GCR technique can bespatially modulated, so as to increase confusion.

In another embodiment of the present invention, a gray componentreplacement fraction in a multiple-illuminant GCR technique may beimplemented to encode an additional, low-resolution image intended forrecovery under illumination by a white light.

Advantages realized in these embodiments include a reduction in the costof a rendered composite image due to the replacement of chromaticcolorants with black colorants, an increase in the dynamic range of thecomposite image, or a reduction in the overall amount of colorantconsumed in rendering the composite image.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 represents reflectance spectra for a white paper substrate andcolorants in the form of Cyan, Magenta, Yellow, and Black dyes (at 100%density) operable in a dye sublimation printer.

FIG. 2 represents the relative radiance spectra for the red, green, blueprimaries generated by a typical cathode ray tube (CRT).

FIG. 3 is a block diagram of systems for spectral multiplexing anddemultiplexing of plural source images, and for rendering a compositeimage having therein at least one encoded source image, constructedaccording to the invention.

FIG. 4 is a simplified schematic diagram of methods operable in thesystem of FIG. 3 for spectrally multiplexing first and second sourceimages in a composite image, rendering the composite image with use ofrespective first and second colorants, and for demultiplexing therendered composite image.

FIG. 5 is a schematic simplified representation of the spectralmultiplexing system of FIG. 3, in which an image processing unit andassociated peripheral devices and subsystems are employed.

FIG. 6 is a simplified schematic representation of the spectraldemultiplexing system of FIG. 3, in which a controller and associatedperipheral devices and subsystems are employed.

FIG. 7 is a schematic representation illustrating the dominance of acyan image subjected to illumination by white light.

FIG. 8 is a schematic representation illustrating the operation of graycomponent replacement (GCR) in the production of a rendered compositeimage, wherein the density of a cyan image when subjected to white lightmay be increased in comparison to the density of the cyan image whensubjected to red light.

FIG. 9 is a rendered composite image, wherein first and second sourceimages were encoded in a composite image and the composite image wasrendered in cyan and yellow colorants, wherein the first and secondsource images are intended for subsequent recovery when subjected to redand blue illuminants, respectively.

FIG. 10 is a rendered composite image created with a 80% GCR fraction,wherein the appearance of the rendered composite image under red andblue illuminants is substantially identical to the appearance of therendered composite image provided in FIG. 9, and wherein the renderedcomposite image appears under white light to be more confused.

FIGS. 11 and 12 are rendered composite images each of which wererendered using GCR in formation of a composite image rendered in cyanand magenta colorants.

FIG. 13 is a rendered composite image wherein GCR was utilized toincorporate first and second source images intended for recovery underblue and red illumination plus a third source image intended forrecovery under white light illumination.

FIG. 14 is a rendered composite image wherein a random variation between0 and 80% in the GCR fraction was implemented over square blocks ofpixels.

DESCRIPTION OF THE INVENTION

Definitions of Terms

Color—A color can be uniquely described by three main perceptualattributes: hue, denoting whether the color appears to have an attributeaccording to one of the common color names, such as red, orange, yellow,green, blue, or purple (or some point on a continuum); colorfulness,which denotes the extent to which hue is apparent; and brightness, whichdenotes the extent to which an area appears to exhibit light. Lightsources used to illuminate objects for viewing are typicallycharacterized by their emission spectrum and to a reduced degree bytheir color temperature, which is primarily relevant forcharacterization off sources with a spectrum similar to a black bodyradiator. See, for instance, Hunt, R. W. G., Measuring Colour, EllisHorwood, 1991, and Billmeyer and Saltzman, Principles of ColorTechnology, 3rd Ed. (Roy S. Berns), John Wiley & Sons, 2000.

Chroma—Colorfulness of an area judged as a proportion of the brightnessof a similarly illuminated area that appears white or transmitting.

Colorant—A dye, pigment, ink, or other agent used to impart a color to amaterial. Colorants, such as most colored toners, impart color byaltering the spectral power distribution of the light they receive fromthe incident illumination through two primary physical phenomenon:absorption and scattering. Color is produced by spectrally selectiveabsorption and scattering of the incident light, while allowing fortransmission of the remaining light. A narrow band (absorbing) colorantexhibits an absorption band that is situated in a substantially narrowregion of the visible region of the spectrum. Cyan, magenta and yellowcolorants are examples of narrow band colorants that selectively absorbred, green, and blue spectral regions, respectively. Some colorants,such as most colored toners, impart color via a dye operable intransmissive mode. Other suitable colorants may operate in a reflectivemode.

Composite Image—An array of values representing an image formed as acomposite of plural overlaid (or combined) colorant image planes. Sourceimages may be encoded as described herein and the resulting image planesare combined to form a composite image. When a rendered composite imageis subjected to a narrow band illuminant, the combined density of allcolorants in the rendered composite image will reveal at least onesource image that is otherwise difficult to distinguish, or the reverse,depending upon the specific colorant/illuminant interaction.

Density (Optical)—The degree of darkness of an image. Higher densityvalues represent greater darkness. Mathematically, optical density isdefined as the negative logarithm of the reflectance or transmittance.The spectral density is correspondingly the negative logarithm of thereflectance/transmittance spectrum.

Hue—Attribute of visual sensation according to which an area appears tobe similar to one of the perceived colors: red, green, yellow, and blue,or to a combination thereof. An achromatic color is a perceived colordevoid of hue and a chromatic color is a perceived color possessing ahue. See, for instance, Fairchild, Mark D., Color Appearance Models,Addison Wesley, 1998.

Gamut—A range of colors; typically, the range of colors that can beproduced by a device.

Grayscale—Image data representing one of a series of tones stepped fromlight to dark.

Gray Component Replacement (GCR)— A technique whereby black ink is usedto replace a portion of common darkness of component colorants.

Image—An image may be described as an array or pattern of pixels thatare mapped in a two-dimensional format. The intensity of the image ateach pixel is translated into a numerical value which may be stored asan array that represents the image. An array of numerical valuesrepresenting an image is referred to as an image plane. Monochromatic orblack and white (gray scale) images are represented as a two-dimensionalarray where the location of a pixel value in the array corresponds tothe location of the pixel in the image. Multicolor images arerepresented by multiple two-dimensional arrays. In a three colorarrangement, each array represents one of the primary colors. In thedigital processing of color images, the individual color separations arerepresented as a digital image with a plurality of discrete elements(“pixels”) defined by position and gray value. In such a system, grayvalue is described as one level in a number of possible states orlevels. When more than two different levels are used in the descriptionof an image, the levels are termed “gray” (without regard to the actualcolor) to indicate that the pixel value is between some maximum andminimum gray level.

Illuminant—Incident luminous energy specified by its relative spectralpower distribution; typically provided by a real or imaginary lightsource having a predefined spectral power distribution. A narrow bandilluminant is an illuminant having a spectral power distribution that issubstantially limited to a narrow region of the spectrum. The bandwidthof the region may vary from extremely narrow for a LASER source, tonarrow band illuminants such as natural or artificial light transmittedthrough a band-limited color filter. Examples of red, green and bluenarrow band illuminants are shown in FIG. 2, which illustrates thespectral power distributions obtained from activated red, green, andblue phosphors in a typical CRT.

Image plane—A two-dimensional representation of image data. Theuppercase letters C, Y, M, K are used herein to indicate two-dimensionalarrays of values representing a monochromatic image or a separablecomponent of a polychromatic (multicolor) image. Two-dimensional arraysof values may also be referred to as “planes.” For example, the Y planerefers to a two-dimensional array of values that represent the yellowcomponent at every location (pixel) of an image.

Imaging Device—A device capable of generating, capturing, rendering, ordisplaying an image; including devices that store, transmit, and processimage data. A color imaging device has the capability to utilize colorattribute information.

Lightness—The perceptual response to luminance; denoted L* and isdefined by the CIE as a modified cube root of luminance.

Primary Colors—Colors, usually three, which are combinable to produce arange of other colors within a color mixing model. All non-primarycolors are mixtures of two or more primary colors. Red, green, and blue(R, G, B) are the standard additive primary colors. Cyan, magenta, andyellow (C, M, Y, K) are the standard subtractive primary colors. Black(K) colorant absorbs light energy substantially uniformly over the fullextent of the visible spectrum and may be added to enhance color andcontrast and to improve certain printing characteristics. Cyan, magenta,and yellow are the subtractive complements of red, green, and blue,respectively and they absorb the light energy in the long, middle, andshort wavelength regions, respectively, of the visible spectrum, leavingother regions of the visible spectrum unchanged. Ideally, the absorptionbands of individual CMY colorants are non-overlapping and completelycover the visible region of the spectrum. Actual CMY colorants do notsatisfy these block-dye assumptions; instead, their absorption spectraare smooth and include some unwanted absorptions in their transmissionbands. The reflectance spectra for white paper and cyan, magenta,yellow, and black colorants (100%) from a dye-sublimation printer areshown in FIG. 1. Red, green, and blue are the additive complements ofcyan, magenta, and yellow respectively.

Saturation—Colorfulness of an area judged in proportion to itsbrightness. Saturation is judged for an area relative to its ownbrightness whereas chroma is judged relative to a similarly illuminatedarea that appears white.

Subtractive Color Model The production of color wherein light issubtracted through a process such as absorption; a color model in whichcolors may be produced by combining various percentages of thesubtractive primaries (cyan, magenta, and yellow).

Introduction to a General Theory of the Invention

Two predominant modes for producing color are: Additive color, wherebycolor is produced by the addition of spectrally selective lights to adark background that is otherwise substantially devoid of light; andsubtractive color, whereby color is produced by spectrally selectivesubtraction of light energy from the light emitted by a source. Red,green and blue lights are typically used as the primaries that are mixedtogether in an additive system. In a subtractive system, colorants aretypically used as the subtractive primaries. These colorants selectivelyabsorb, or subtract, a portion of the visible spectrum of incident lightwhile transmitting the remainder. Cyan, Magenta, and Yellow colorantsare typically used.

Color in printed images results from the combination of a limited set ofcolorants deposited on a substrate over a small area in densitiesselected to integrate the desired color response. This is accomplishedin many printing devices by reproducing so called “separations” of theimage, where each separation provides varying gray values of a singleprimary color. When the separations are combined together, the result isa full color image.

Colorants that are deposited on a reflective substrate, such as a papersheet, will selectively transmit incident light in a first pass to thesurface of the substrate whereupon the transmitted light is thenreflected by the substrate and is again filtered by the colorants in asecond pass, thus encountering additional selective absorption beforebeing perceptible as a particular color by an observer. It is alsocommon for colorants to possess a degree of scattering, and the colorappearance of a colorant on a printed substrate is determined by theamount and types of the colorants present, and the combination of theirabsorption and scattering properties.

In the practice of the invention, most colors in a subtractive colorsetting may be reproduced in an image by use of different proportions ofcyan, magenta, and yellow colorants. Each of these colorants ischaracterized primarily by its absorption characteristics. An idealizedcyan colorant, for instance, may be defined as having absorption bandcovering the wavelength interval 600–700 nm, commonly referred to as thered region of the spectrum. Likewise, an idealized magenta colorant hasan absorption band covering the interval 500–600 nm, commonly referredto as the green region of the spectrum, and an idealized yellow coloranthas an absorption band covering the interval 400–500 nm, commonlyreferred to as the blue region of the spectrum.

Thus, cyan, magenta, and yellow colorants absorb red, green and bluelight, respectively. The idealized absorption bands for the cyan,magenta, and yellow colorants are referred to as the block-dyeassumption. In reality, colorants exhibit significant deviations fromthis idealized behavior, including variations of absorption within theabsorption band, extension of the absorption band beyond the idealizedlimits, and scattering in the colorants. In particular, the absorptionof light in a spectral region outside the main absorption band of acolorant (as, for example, demonstrated by absorption in the blue andred regions of the spectrum by a magenta colorant), is consideredunwanted absorption. Among typical colorants used for CMYK printing,magenta demonstrates the most unwanted absorptions and yellow the least.The black colorant absorbs uniformly through the visible region of thespectrum and can be used as a replacement for combined CMY for reasonsof economy and improved rendition of dark regions.

Thus, according to the subtractive principle, a surface layer of asubstrate such as a sheet of white paper, on which one can vary theconcentrations of a cyan, a magenta, and a yellow colorant, therebyoffers the means of varying the intensities of the reddish, greenish,and bluish parts of the white light reflected from the paper. To producea subtractive color image reproduction, one can control theconcentrations of the three colorants independently at localized areason the paper substrate.

All surfaces, whether of a colorant or substrate, reflect from theirtopmost layer a certain proportion of the incident light which is addedto that reflected from the body of the surface. This light reflectedfrom the topmost layer is the same color as the illuminant, andtherefore when a color surface is viewed in white light, some of thewhite light is added to the colored light reflected from the body of thesurface and the colorfulness is therefore reduced. Most surfaces alsoexhibit some degree of gloss, and this means that, if the lighting isdirectional, the white light reflected from the topmost layer of thesurface will be confined chiefly to a single direction, thus alteringthe appearance of the image to some extent depending on the angles ofviewing and illumination.

Under normal viewing illumination, the eye adapts to the white-point,which usually corresponds to blank paper with the highest reflectanceand different colors can be seen by the eye for prints made withdifferent colorant combinations. However, under relatively narrow bandillumination, such as that obtained from a phosphor excited by a singlegun of a CRT monitor, the eye is unable to distinguish color. Imagesviewed under narrow band illumination therefore appear to have onlyvarying levels of gray and little or no chroma. Since the absorptioncharacteristics of each of a plurality of colorants will differ indifferent spectral bands, the respective reflectance (or density) ofeach colorant when subjected to a series of differing narrow bandilluminants will also appear to have varying levels of gray.

The present invention accordingly exploits the interaction betweencertain narrow band illuminants and their corresponding (complementary)colorants (especially the colorants typically used for printing), andthe manner in which the eye detects images illuminated with illuminantshaving narrow band spectral power distributions. The methodologydescribed herein may be generalized to apply to an arbitrary number ofilluminants and colorants, and for the purpose of simplicity theinvention is described with reference to the cyan, magenta, yellow, andblack colorants commonly used in color printing applications, and to thenarrow band red, green, and blue illuminants commonly generated byCRT-based light sources. This description thus makes reference to thehandling of monochromatic and color source images encoded according toan array of colorants such as the CMYK color primaries. However, it willbe apparent to one of ordinary skill in the art that there arealternative spectral schemes to be employed in the spectral multiplexingof the invention. An alternative would include a color system thatemploys primary colorants other than CMYK for color representations,such as systems that use RGB primaries or high-fidelity colorants suchas orange and green. Still another alternative would be to employ theinvention in a system that processes different types of multi-spectraldata, such as source images encoded with respect to narrow bandcolorants responsive to illuminants generated from ultraviolet orinfrared light sources.

As the present invention is directed to the multiplexing ordemultiplexing of at least one source image encoded in a compositeimage, the composite image may be defined in a spectrally multiplexed(SM) image plane. This plane may have any number of different patternsof pixels, with a primary characteristic being that the plane isspectrally multiplexed. In general, at each location in the SM plane, apixel value representing one or more spectral components may be present,and which spectral component is present depends on the gray level of thecorresponding pixel in one of the source image planes. (The inventionmay also have applications to SM planes in which each pixel includescolor values representative of color separation image data from morethan one source image plane.)

The general theory of the invention may be understood with reference toa rendering device in the form of a color hardcopy output device, suchas a printer, and to a mathematical framework that employs nomenclaturesimilar to that used in conventional color imaging. Consider a colorhardcopy output device with M colorants. Prints from this device are tobe viewed under N different illuminants. The luminance characterizationof the output device under the N illuminants

{L_(i)}_(i = 1)^(N)is given by the relation between the control values

{A_(j)}_(j = 1)^(M)used for each of the M colorants at a given pixel location and theluminance produced at the given pixel location under each of the Nilluminants. This can be denoted as the set of N functions, where i=1,2,. . . N:f(A ₁ , A ₂ , . . . A _(M))=luminance of region

-   -   with colorant control values A₁, A₂, . . . A_(M) under ith        illumination L_(i)

In the following description, we assume that a control value of 0 for agiven colorant represents no printing of that colorant. This conventionis not a requirement for the invention and is only adopted fornotational simplicity.

The description herein is limited to the case of luminancecharacterization alone, because under narrow band illumination the eyeprimarily sees differences of luminance and is unable to distinguishmost color differences. Note that luminance as described here agrees inconcept with its standard usage, i.e., as a measure of the perceivedlight energy; however, it's definition is not limited to theconventional usage and is expanded to comprehend the special viewingsituations also described herein. In particular, under narrow bandillumination, specific visual effects may influence the perception of asource image. A specific instance of this is the Purkinje effect thatcauses increased sensitivity in the blue region of the spectrum at lowlight levels, which may be of particular relevance for viewing underblue light and CRT illumination in general. Some of the advancedconcepts from photometry and colorimetry that are required in suchsituations are described for instance in G. Wyszecki and W. S. Stiles,Color Science: Concepts and Methods, Quantitative Data and Formulae,2^(nd) Edition, John Wiley and Sons (1982).

The methods of the present invention are directed to the multiplexing,rendering, and recovery via demultiplexing of a source image encoded ina composite image. We assume that the one or more source images to berecovered are described by the spatial luminance distributions desiredunder each of the illuminants (although, in the alternative, any otherequivalent specification that can be transformed to luminance/densitymay be used.) Thus, there are N images specified, with Y_(i)(x, y) beingthe desired luminance values that we wish to produce under the ithilluminant L_(i) where x, y denote the two spatial coordinates. For thepurposes of simplifying the notation in the following discussion, thespatial dependence is sometimes dropped in the following descriptionwith the understanding that the discussion applies to each pixellocation independently.

To examine the basic methodology symbolically, consider a simplifiedexample of a composite image rendered in cyan and yellow colorants. Inthe simplified example below, additivity of “RGB” densities is assumed.This is for the purposes of simple illustration of the principles onlyand not intended to restrict the invention; in those situations wherethis approximation is invalid, more precise assumptions can be made. Inthis example: C, M, Y, K and R, G, B will respectively denote thecolorants and illuminants; a superscript will denote illuminant; and asubscript will denote a colorant. Let:

-   -   d^(R)=density of the image perceived under R illumination,    -   d^(B)=density of the image under B,    -   d_(C) ^(R)=density C separation under R,    -   d_(C) ^(B)=density C separation under B,    -   d_(Y) ^(R)=density Y separation under R,    -   d_(Y) ^(B)=density Y separation under B.

When illuminated with a R or B illuminant, the total density perceivedcan be approximated as,d ^(R)(x, y)=d _(C) ^(R)(x, y)+d _(Y) ^(R)(x, y)≈d _(C) ^(R)(x, y)d ^(B)(x, y)=d _(C) ^(B)(x, y)+d _(Y) ^(B)(x, y)≈d _(Y) ^(B)(x, y)

Accordingly, this methodology exploits the characteristically lowdensity of a colorant when subjected to a first illuminant and thecharacteristically high density exhibited by the same colorant whensubjected to a second, differing illuminant. Thus, at least oneperceptibly distinct source image (that is encoded in the renderedcomposite image by use of the particular colorant), will beimperceptible (or nearly so) to an observer when subjected to the firstilluminant, but perceptibly distinguishable to an observer whenilluminated by the second illuminant. Upon perception of the sourceimage by an observer, the source image may be comprehended and theinformation embedded in the composite image, or the composite imageitself, is thereby readily comprehended.

Determinations of Gamut Mapping, Dynamic Range, and Colorant Interaction

The example presented above assumed that colorant interactions can beentirely ignored. This assumption is not true with most practicalcolorants and additional considerations are therefore required.

Consider the case of a rendered composite image that is produced byusing C and M colorants for subsequent illumination under red and greenilluminants. For simplicity, in our illustration below we assumeadditivity for the red, green, blue band densities, as the general casefor situations where this approximation does not hold is describedsubsequently. A first source image may be recovered primarily from thecyan component of a composite image, and a second source image may berecovered primarily from the magenta component; however, unwantedabsorption by these colorants are preferably compensated to avoidartifacts discernible by an observer. The total density under redillumination at pixel location (x,y) can be approximated asd ^(R)(x,y)=d _(C) ^(R)(x,y)+d _(M) ^(R)(x,y)

and the total density under green illumination isd ^(G)(x,y)=d _(M) ^(G)(x,y)+d _(C) ^(G)(x,y)

where d_(U) ^(V) (x,y) represents the visual density under illuminant Vdue to colorant U at pixel location (x,y).

The terms d_(M) ^(R)(x,y) and d_(C) ^(G)(x,y) represent the unwantedabsorption. In the simplest case, it can be assumed that a colorant'sabsorption under its complementary illuminant is used for twopurposes: 1) to recover the desired image and 2) to compensate forunwanted absorption by the other colorant(s) present in the compositeimage. So a magenta colorant may be used to produce the desired image tobe seen under green illumination and to compensate for the unwantedabsorption of the cyan colorant; a cyan colorant may be used to producethe desired image under red illumination and to compensate for unwantedabsorption of magenta under red illumination.

The portion that is used to compensate for the unwanted absorptionshould combine with the unwanted absorption to result in a constantspatial density so as to make it “disappear”. Let d1_(C) ^(R)(x,y)represent the portion of Cyan density that is used to compensate for theunwanted absorption of Magenta under red, which is determined byd1_(C) ^(R)(x,y)+d _(M) ^(R)(x,y)=constant=q ^(R)

The remaining density contribution of Cyan under red illumination isd2_(C) ^(R)(x,y)=d_(C) ^(R)(x,y)−d1_(C) ^(R)(x,y). Note that the totaldensity can be written in terms of these components as

$\begin{matrix}{{d^{R}\left( {x,y} \right)} = {{{d_{c}^{R}\left( {x,y} \right)} + {d_{M}^{R}\left( {x,y} \right)}} = {{{d2}_{C}^{R}\left( {x,y} \right)} + \left( {{{d1}_{C}^{R}\left( {x,y} \right)} + {d_{M}^{R}\left( {x,y} \right)}} \right)}}} \\{= {{{d2}_{C}^{R}\left( {x,y} \right)} + q^{R}}}\end{matrix}$

Therefore the overall visual density under red illumination correspondsa constant background density of q^(R) with the spatially varyingdensity pattern of d2_(C) ^(R)(x,y) superimposed. This spatially varyingpattern is the one that is seen under red illumination and shouldtherefore represent the first multiplexed image that is to be seen underred illumination.

In a similar manner the density contribution of magenta under greenillumination can be decomposed into a component d1_(M) ^(G)(x,y) that isused to compensate for the unwanted absorption of cyan under greenillumination, given byd1_(M) ^(G)(x,y)+d _(C) ^(G)(x,y)=constant=q ^(G)

and the remaining componentd2_(M) ^(G)(x,y)=d _(M) ^(G)(x,y)−d1_(M) ^(G)(x,y)

which satisfies

$\begin{matrix}{{d^{G}\left( {x,y} \right)} = {{{d_{M}^{G}\left( {x,y} \right)} + {d_{C}^{G}\left( {x,y} \right)}} = {{{d2}_{M}^{G}\left( {x,y} \right)} + {{d1}_{M}^{G}\left( {x,y} \right)} + {d_{C}^{G}\left( {x,y} \right)}}}} \\{= {{{d2}_{M}^{G}\left( {x,y} \right)} + q^{G}}}\end{matrix}$

Therefore the overall visual density under green illuminationcorresponds to a constant background density of −q^(G) with thespatially varying density pattern of d2_(C) ^(R)(x,y) superimposed. Thisspatially varying pattern is the one that is seen under red illuminationand should therefore represent the second multiplexed image that is tobe seen under green illumination.

Since the terms d2_(C) ^(R)(x,y) and d2_(M) ^(G)(x,y) represent thevisual variations in density corresponding to the two multiplexedimages, we would like to maximize their dynamic range. Since colorantscan only add positive density, this requirement translates to minimizingthe terms q^(R) and q^(G) subject to meeting the required equations andthe physical constraint that colorants can only add positive density. Wewould therefore like to determine the smallest feasible values of q^(R)and q^(G) for which the above equations are feasible.

For the purpose of further illustration we use a first orderapproximation, that the amount of colorant added to compensate forunwanted absorption of the other colorant, itself only contributes anegligible amount of unwanted absorption (because of its small value).This assumption implies that the component of Magenta used to offsetunwanted absorption of Cyan contributes negligibly to unwantedabsorption under green and the component of Cyan used to offset unwantedabsorption of Magenta contributes negligibly to unwanted absorptionunder blue. This assumption is used for illustration only, in practice,one can iteratively determine the appropriate amounts to account forhigher-order effects or use an appropriate model/LUT as outlinedsubsequently in this disclosure. With this simplifying assumption, therange achievable for the desired spatially varying pattern d2_(C)^(R)(x,y) under red illumination is between q^(R) and d_(C) ^(R)(x,y)with a total density variation or dynamic range of d_(C)^(R)(x,y)−q^(R). Likewise the total density range available under greenillumination is d_(M) ^(G)(x,y)−q^(G).

One set of feasible values for the terms q^(R) and q^(G) can bedetermined as:q ^(R)=max(d _(M) ^(R)(x,y))=d _(M) ^(R)(255)=max density for Magentaunder red illuminantq ^(G)=max(d _(C) ^(G)(x,y))=d _(C) ^(G)(255)=max density for Cyan undergreen illuminant

This can be thought of as follows: the background density under redlight q^(R) is equal to the maximum unwanted density that one can havefrom Magenta. The Cyan density component d1_(C) ^(R)(x,y) is designedcarefully so that the combination of Cyan and Magenta at each pixel hasa density q^(R), this can be achieved by putting no Cyan where Magentais 100% (255 digital count) and appropriate amounts of Cyan to make upthe density to q^(R) at pixels which have less than 100% Magenta. Asimilar argument applies to the Magenta density component d1_(M)^(G)(x,y) that compensates for the unwanted absorption of Cyan under redillumination.

With the notation and terminology defined earlier, the generalmulti-illuminant imaging problem reduces to the following mathematicalproblem:

Given N luminance values

{Y_(i)}_(i = 1)^(N)corresponding to the desired luminance values under the N differentilluminants, determine a set of control values for the M colorants

{B_(j)}_(j = 1)^(M)to be used in printing a pixel, such that for all i=1,2, . . . N:f _(i)(B ₁ , B ₂ , . . . B _(M))=luminance of pixel under ithillumination L _(i) =Y _(i)  (1)

Typically, for N>M (number of image specifications>number of colorants)the system is over-determined and has a solution only under severeconstraints on the

{Y_(i)}_(i = 1)^(K)luminance values limiting its utility in illuminant multiplexed imaging.Even if N≦M (number of image specifications≦number of colorants), thesystem of N equations presented in (1) above has a solution(corresponding to realizable device control values

{B_(j)}_(j = 1)^(M))only in a limited region of luminance values, which we refer to as thegamut for the spectrally multiplexed imaging problem:

$\begin{matrix}\begin{matrix}{G = {{gamut}{\mspace{14mu}\;}{achievable}\mspace{20mu}{for}{\mspace{11mu}\mspace{11mu}}{illuminant}}} \\{{multiplexed}\mspace{14mu}{imaging}} \\{= \left\{ {Y\;\varepsilon\; R_{+}^{K}\mspace{20mu}{such}\mspace{14mu}{that}{\mspace{14mu}\;}{{system}{\mspace{11mu}\;}(1)}} \right.} \\{\left. {{has}\mspace{20mu} a\mspace{14mu}{realizable}\mspace{14mu}{solution}} \right\}\;}\end{matrix} & (2)\end{matrix}$

where Y=[Y₁, Y₂, . . . Y_(N)], denotes the vector of luminance valuesunder the N illuminants, and R₊ is the set of nonnegative real numbers.For specified N-tuples of luminance values within the gamut G, there isa set of realizable control values such that a pixel printed with thecontrol values produces the required luminance values under the givenilluminants. Vice versa, N-tuples of luminance values outside the gamutG cannot be created using any realizable control values. The situationis analogous to the limited color gamut encountered in colorreproduction. It is necessary to include a gamut mapping step in thespectral multiplexing described herein to ensure that the source imagesare limited to the gamut of the system before attempting to reproducethem. The gamut mapping may be image independent or image dependent,where the term image is used to imply the set of desired source imagesrecoverable under the different illuminants. In addition, the set ofimages to be multiplexed may be designed to take into account the gamutlimitations and produce the best results with those gamut limitations.

Once the source images to be multiplexed have been mapped to theachievable gamut G, the problem of reproduction reduces to thedetermination of the control values for each of the M colorants for eachpixel. This corresponds to an inversion of the system of equations in(1) and in a manner similar to color calibration, the inverse could bepre-computed and stored in N-dimensional look-up tables (LUTs), with oneLUT per colorant (or alternately, a single N-dimensional LUT with Moutputs).

In practice, the function in (1) itself needs to be determined throughmeasurements of the device response by printing a number of patches withdifferent M-tuples of control values and measuring them suitably toobtain the luminance under the different illuminants. The full spectrumof the patches may be measured for instance on a spectrophotometer fromwhich the luminances may be computed using the spectral powerdistribution of the different illuminants and the visual luminancesensitivity function. The visual luminance sensitivity function mightincorporate adjustments for the appropriate light level that account forvisual phenomena such as the Purkinje effect. See for instance, G.Wyszecki and W. S. Stiles, Color Science: Concepts and Methods,Quantitative Data and Formulae, 2^(nd) Ed., 1982, John Wiley and Sons,Inc., New York, N.Y., in particular pages 406–409.

Simplification According to a One Illuminant/One Colorant InteractionAssumption

Several simplifications can be incorporated into the general frameworkabove. Suppose first, that N=M and the colorants and lights are suchthat colorant i absorbs only illuminant L_(i) and is completelytransparent to all other colorants, then we have

$\begin{matrix}\begin{matrix}{{{f_{i}\left( {A_{1},A_{2},{\ldots\mspace{14mu} A_{M}}} \right)} = {{{function}\mspace{14mu}{of}\mspace{14mu} A_{i\;}\mspace{14mu}{alone}\mspace{45mu} i} = 1}},2,{\ldots\mspace{14mu} N}} \\{= {{f_{i}\left( {0,0,\ldots\mspace{11mu},0,A_{i},0,{\ldots\mspace{11mu} 0}} \right)} \equiv {g_{i}\left( A_{i} \right)}}}\end{matrix} & (3)\end{matrix}$

The system of equations in (1) then reduces to M independent nonlinearequations one for each colorant under the corresponding illumination:g _(i)(B _(i))=Y _(i) i=1,2, . . . N  (4)

The achievable gamut can be defined as follows. Let:

$\begin{matrix}{g_{i}^{\min} = {\min\limits_{A_{i}}{g_{i}\left( A_{i} \right)}}} \\{g_{i}^{\max} = {\max\limits_{A_{i}}{g_{i}\left( A_{i} \right)}}} \\{h_{i} = \left\lbrack {g_{i}^{\min},g_{i}^{\max}} \right\rbrack} \\{= {{the}\mspace{14mu}{interval}\mspace{20mu}{of}\mspace{14mu}{luminances}}} \\{{from}\mspace{14mu} g_{i}^{\min}\mspace{14mu}{to}\mspace{14mu} g_{i}^{\max}}\end{matrix}$where i=1,2, . . . N  (5)G ₁=achievable gamut under assumption of one illuminant interacting withonly one colorant=h ₁ ×h ₂ × . . . ×h _(N)  (6)

In other words, the achievable gamut is the product set of theseindividual luminance intervals. Note that the assumption in Eq. (6) isthat the complete interval between the max and min limits can berealized without any “gaps” which would typically be expected withphysical colorants. (For a definition of a product set, see forinstance, Friedman, The Foundations of Modern Analysis, Dover, 1982, NewYork, N.Y.)

Under the assumption of one illuminant interacting with only onecolorant, the multi-illuminant imaging characterization problem reducessignificantly. Instead of requiring N-dimensional LUTs onlyone-dimensional LUTs—one per colorant are needed. The value of eachcolorant may be determined by the luminance under the correspondingillumination alone.

Alternative Simplifications

In practice, the assumption of one illuminant interacting with only onecolorant does not hold for typical colorants. However, if the strongestinteractions are between the ith illuminant and the ith colorant withother interactions having a smaller magnitude, the achievable gamut is areduced N-dimensional region that is contained in G₁. Note that thesituation of using cyan, magenta, and yellow colorants with red, green,and blue lights for illumination corresponds to this case, where thecyan interacts most with red, magenta with green, and yellow with blue.Note also that the use of a black colorant that (typically) absorbs allilluminants almost equally, does not satisfy the requirement of stronginteraction with only one illuminant. In practice this implies that ablack colorant should be viewed as an additional colorant, i.e., if onecolorant is black we should have:N=number of illuminants=number of images≦number of colorants−1=M−1

Black may, however, be used with other colorants in special situations(as is described in the examples below) and can help improve achievablegamut (i.e., improve dynamic range), simplify computation, and reducecost.

Simplifications Based on Additive Density Models

The general technique described earlier requires a measurement of thedevice response in the M-dimensional input space of device controlvalues, and the final characterization may be embodied in the form ofmulti-dimensional LUTs with N-dimensional inputs. In several cases, themeasurement and storage/computation requirements for multi-illuminantcolor imaging can be significantly reduced by using simple models of theoutput processes. One useful model is to assume that the visualdensities follow an additive model, i.e.,

$\begin{matrix}{{{{d_{i}\left( {A_{1},A_{2},{\ldots\mspace{11mu} A_{M}}} \right)} \equiv {{- \log}\mspace{11mu}\left( \frac{f_{i}\left( {A_{1},A_{2},{\ldots\; A_{M}}} \right)}{f_{i}\left( {0,0,{\ldots\mspace{11mu} 0}} \right)} \right)}} = {{- {\sum\limits_{j = 1}^{M}\;{\log\mspace{11mu}\left( \frac{f_{i}\left( {0,0,\ldots\mspace{11mu},A_{j},{\ldots\mspace{14mu} 0}} \right)}{f_{i}\left( {0,0,{\ldots\mspace{14mu} 0}} \right)} \right)}}} = {- {\sum\limits_{j = 1}^{M}\;{d_{i}\left( A_{j} \right)}}}}}{where}} & (7) \\{{d_{i}\left( A_{j} \right)} \equiv {{- \log}\mspace{11mu}\left( \frac{f_{i}\left( {0,0,\ldots\mspace{11mu},A_{j},{\ldots\mspace{14mu} 0}} \right)}{f_{i}\left( {0,0,{\ldots\mspace{14mu} 0}} \right)} \right)}} & (8)\end{matrix}$

(Traditionally, densities are defined as logarithms to the base 10, anyother base can also be used in practice as it changes the densities onlyby a scale factor and does not impact any of the other mathematicaldevelopment.) Note as per our convention, the control values {0,0, . . ., 0} represent an blank paper substrate and therefore f_(i)(0,0, . . .0) represents the luminance of the paper substrate under the ithilluminant, and the logarithmic terms represent paper normalized visualdensities. The additive model for visual densities is motivated by theBeer-Bouguer law for transparent colorant materials and the assumptionof relatively narrow band illumination, for which the additive nature ofspectral density implies the approximation above is a valid one. Themodel also often provides a reasonable approximation for halftone mediawhere the assumptions do not strictly hold. (For a more detailedbackground, see: F. Grum and C. J. Bartleson, Ed., Optical RadiationMeasurements: Color Measurement, vol. 2, 1983, Academic Press, New York,N.Y. or G. Sharma and H. J. Trussell, “Digital Color Imaging”, IEEETransactions on Image Processing, vol. 6, No. 7, pp. 901–932, July1997.) Full computations using a spectral density model might beperformed if necessary to improve the model accuracy, this would bepotentially advantageous in a situation where the illuminating lightsare not strictly narrow band. The terms

${d_{i}\left( A_{j} \right)} \equiv {\log\mspace{11mu}\left( \frac{f_{i}\left( {0,0,\ldots\mspace{11mu},A_{j},{\ldots\mspace{14mu} 0}} \right)}{f_{i}\left( {0,0,{\ldots\mspace{14mu} 0}} \right)} \right)}$represent the paper normalized visual density of a patch printed withthe jth colorant alone and no other colorants, with the control valuefor the jth colorant set as A_(j). Therefore the additive density modelproposed above allows the determination of the visual density of anypatch based on the visual density of control patches of individualcolorants. This reduces significantly the number of measurementsrequired. Measurements of “step-wedges” of the individual colorants (forwhich other colorants are not printed) allow one to determine thefunctions d_(i)(A_(j)) i=1,2, . . . N, j=1,2, . . . M, from which thecomplete device characterization function can be determined using Eq.(8).

Using the above model, the system of equations in (1) reduces to:

$\begin{matrix}{{\sum\limits_{j = 1}^{M}\;{d_{i}\left( B_{j} \right)}} = {{\log\mspace{11mu}\left( {Y_{i}\text{/}Y_{i}^{0}} \right)\mspace{14mu}{where}\mspace{20mu} Y_{i}^{0}} = {f_{i}\left( {0,0,{\ldots\mspace{14mu} 0}} \right)}}} & (9)\end{matrix}$where Y _(i) ⁰ =f _(i)(0,0 . . . 0)  (9)

The equations in (9) represent a system of K nonlinear equations in Mvariables (B₁, B₂, . . . B_(M)). The functions d_(i)(A_(j)) areavailable from the measurements of the “step-wedges” and the aboveequations can be solved for the control values B_(j) for luminancevalues within the gamut G, which was defined earlier. For points outsidethe gamut, the equations may be solved in an approximate sense providinga (less-controlled) form of gamut mapping.

Further simplification of these equations is possible by assuming thatthe densities in different spectral bands are linearly related, i.e.,d _(i)(C)=α_(i) ^(j) d _(j)(C) i=1,2, . . . N  (10)

where α_(i) ^(j)=d_(i)(C)/d_(j)(C) is the proportionality factorrelating the visual density for the jth colorant under the ithilluminant to the visual density for the jth colorant under the jthilluminant and is assumed to be independent of the colorant value C, andα_(j) ^(j)=1, Thus the convention adopted in Eq. (10) is that thedensity of the jth colorant under all other illuminants is referenced toits density under the jth illuminant itself, which is not strictly arequirement of our model but is chosen because it results in asimplification of the notation alternate conventions could also beequivalently used. Equation (10) is also motivated by the Beer-Bouguerlaw for transparent colorant materials and the assumption of relativelynarrow band illuminants. (For a more detailed background, refer to: F.Grum and C. J. Bartleson, Ed., Optical Radiation Measurements: ColorMeasurement, vol. 2, 1983, Academic Press, New York, N.Y. or G. Sharmaand H. J. Trussell, “Digital Color Imaging”, IEEE Transactions on ImageProcessing, vol. 6, No. 7, pp. 901–932, July 1997.) Even though a numberof colorants and marking processes do not follow the Beer-Bouguer lawexactly, in practice, Eq. (10) often provides a reasonably accurateempirical model for measured data and may be used for the purposes ofthe current invention. With the simplification of (10) the system ofequations in (9) reduces to a linear system of equations:

$\begin{matrix}{{{\sum\limits_{j = 1}^{M}\;{\alpha_{i}^{j}{d_{j}\left( B_{j} \right)}}} = {{{\log\left( {Y_{i}\text{/}Y_{i}^{0}} \right)}\mspace{14mu} i} = 1}},2,{\ldots\mspace{11mu} N}} & (11)\end{matrix}$

which can be written in matrix-vector notation asAd=t  (12)

where A is the N×M matrix whose ijth element is α_(i) ^(j), d is M×1 thevector whose jth component is d_(j)(B_(j)) and t is the N×1 vector whosejth component is log(Y_(i)/Y_(i) ⁰).

The system of linear equations can be solved to determine a value of d,which provides the desired luminance values under the differentilluminants (corresponding to the multiplexed images). The individualcomponents of d, i.e., the d_(j)(B_(j)) values can then be used with thevisual density response for the jth colorant under the jth illuminant todetermine the control value corresponding to the jth colorant, i.e.,B_(j). This process is analogous to inverting a one-dimensional tonereproduction curve (1-D TRC). Repeating the process for each colorantprovides the complete set of colorant control values required by

{B_(j)}_(j = 1)^(M)that produce the desired set of luminance values under the differentilluminants.

Note that if N=M, the above set of equations has a unique solutionprovided A is invertible, which is normally the case for typicalcolorants and illuminants. The solution in this case is obtained simplyby inverting the matrix A. Furthermore, if the colorants and illuminantscan be ordered in correspondence, i.e., colorant i absorbs illuminant imost and the other illuminants to a lesser extent, then α_(i) ^(j)≦α_(j)^(j)=1, for all i=1,2 . . . N, i.e., the matrix A is square with theelements along the diagonal as the largest along each row, which isoften desirable from a numerical stability standpoint. If M>N the systemof equations will have multiple mathematical solutions, and the choiceof a particular solution may be governed by additional criteria. Oneexample of a criterion for choosing among the multiple mathematicalsolutions is feasibility, a feasible solution being a set of densityvalues that can be realized with the range of colorant control valuesexercisable.

The model inherent in Eq. (12) can also be used to determine suitableapproximations to the achievable gamut G and can be of assistance inperforming gamut mapping. Typically, the density curves d_(j)(C) aremonotonically increasing functions of the colorant control value C andthe achievable range of densities for the jth colorant under the jthilluminant is between

d_(j)^(min) = d_(j)(0) = 0  and    d_(j)^(max) = d_(j)(C_(j)^(max)), where  C_(j)^(max)is the maximum control value for the jth colorant. The achievable gamutassuming the model of Eq. (12) is valid is

$\begin{matrix}\begin{matrix}{G_{D} = {{achievable}{\mspace{11mu}\;}{luminance}\mspace{20mu}{gamut}}} \\{{assuming}{\mspace{14mu}\;}{additive}{\mspace{11mu}\;}{densities}} \\{= \begin{Bmatrix}{y\mspace{20mu}{such}\mspace{14mu}{that}\mspace{20mu}{there}\mspace{20mu}{exists}\mspace{14mu} a\mspace{14mu} d} \\{{{with}\mspace{14mu}{Ad}} = {{\log\left( {y\text{/}y^{0}} \right)}{and}}} \\{0 = {d^{\min} \leq d \leq d^{\max}}}\end{Bmatrix}}\end{matrix} & (13)\end{matrix}$

where d^(min) is an M×1 vector whose jth component is

d_(j)^(min) = 0,and d^(max) is an M×1 vector whose jth component is

d_(j)^(max),y is an N×1 vector whose ith component represents the luminance underthe ith illuminant L_(i), and y⁰ is a N×1 vector whose ith componentrepresents the paper luminance under the ith illuminant. Theinequalities, the division, and the logarithm in the right hand side ofEq. (13) are understood to be applicable on a term-by-term basis for thevectors.

The N images to be produced under the N illuminants provide a N-tuplefor each pixel location corresponding to the desired luminance values atthat pixel location under the N illuminants. The N-tuples correspondingto all the pixel locations must lie within the gamut G defined earlierin order for the image to be producible using the given colorants andilluminants. If images specified for multiplexing do not satisfy thisconstraint some form of gamut mapping is necessary.

A simple image-independent gamut mapping scheme may be defined asfollows. First, ranges of luminance values under the differentilluminants are determined such that all possible values within theseranges lie within the gamut G. This is mathematically equivalent tostating that we determine a set of N-intervals

S_(i) = [Y_(i)^(min), Y_(i)^(max)],i=1,2, . . . N such that the product set of these intervals is containedwithin the gamut G, i.e.,S₁×S₂×S₃× . . . ×S_(N) ⊂G  (14)

The gamut mapping may then be performed on an image independent basis bymapping the set of requested luminance values under the ith illuminantto the interval

S_(i) = [Y_(i)^(min), Y_(i)^(max)]by some (typically monotonous) function. The interval S_(i) determinesthe luminance dynamic range achieved under the ith illuminant. Sincethere are typically multiple choices of the sets

{S_(i)}_(i = 1)^(N)for which Eq. (14) is valid, one method for selecting the intervals maybe by using a max min optimization where we maximize the minimum dynamicrange achievable. Mathematically, this approach can be described asfollows: Select the sets

{S_(i)}_(i = 1)^(N)such that min_(i) f(S_(i)) is maximized, where f(S_(i)) is some suitablychosen function that measures the contrast achieved corresponding to theluminance range S_(i). Examples of suitable choices of the function f( )are simple luminance ratio i.e.,

f(S_(i)) = Y_(i)^(max)/Y_(i)^(min),or density range

f(S_(i)) = log (Y_(i)^(max)/Y_(i)^(min)),or CIE lightness range

f(S_(i)) = L^(*)(Y_(i)^(max)) − L^(*)(Y_(i)^(min)),where L( ) is the CIE lightness function. (See for instance, G. Wyszeckiand W. S. Stiles, Color Science: Concepts and Methods, Quantitative Dataand Formulae, 2^(nd) Ed., 1982, John Wiley and Sons, Inc., New York,N.Y.) Note that the choice of the density range as the function in themax-min optimization along with the model of Eq. (13) reduces this to alinear max-min optimization problem with box constraints that can besolved using numerical optimization schemes.

Illustrated Embodiments of the Invention

For a general understanding of the present invention, reference is madeto the drawings. In the drawings, like reference numerals have been usedthroughout to designate identical elements.

FIG. 3 illustrates a system 100 operable in a first mode for spectrallymultiplexing a plurality of source images to form a composite image, ina second mode for rendering the composite image, or in a third mode fordemultiplexing the spectrally multiplexed composite image so as torecover at least one of the plurality of source images for advantageousviewing by an observer.

As shown in FIG. 3, a plurality of disparate source image arrays 11-1,11-2, . . . 11-N are presented to an image input device 20 in a spectralmultiplexing system 101. Image input device 20 may be equipped toreceive plural monochromatic images or a combination of monochromaticand multichromatic images. Image input device 20 may include an imagecapture device such as a digital scanner coupled to a random accessmemory, or any type of analog or digital camera coupled to a storagemeans such as a computer memory or a magnetic or optical recordingmedium. Image input device 20 may also include means for receiving animage that had previously been stored in a random access memory, onvideo tape, or a laser-encoded disk, etc., or for receiving an imagecreated by a computer image generator, or an image encoded in anappropriate format and transmitted on a network.

The illustrative representation of the plural source images inrespective image arrays received by the image input device 20 in thisexample includes a first source image 12-1 represented in a first sourceimage array 11-1 and a second source image 12-2 represented in a secondsource image array 11-2. The system 101 can optionally receive N sourceimages which are represented in a respective image arrays. In thisexemplary embodiment of the invention, disparate pictorial source imagesare employed and at least one of the plural source images is intendedfor ultimate recovery (via spectral demultiplexing) from a compositeimage.

Once the source image data is received in the input image device 20, itis presented to a spectral multiplexer 30, which encodes a datarepresentation of a composite of at least the first and second sourceimages, so as to provide a composite image 32 on an spectrallymultiplexed (SM) image plane. Such encoding may proceed in oneembodiment with mapping for every pixel location, or by mapping inlocalized areas rather than specific pixels, to the composite image 32,so as to multiplex the information necessary for encoding of eachcorresponding pixel located in each source image.

Next, according to operation of a composite image rendering system 102,data representative of the composite image is provided to a renderingdevice 40, which can be connected to the spectral multiplexer 30 by anyone of a variety of suitable means for transmitting or storingelectronic information. The rendering device 40 records the compositeimage 32 on a substrate 44 with use of a predetermined array of narrowband colorants, so as to form a rendered composite image 42. Therendered composite image 42 is thereby fixed on the substrate 44.

The rendered composite image 42 is available for viewing in ambientlight by an observer 70. Although the rendered composite image 42 isrepresentative of data encoded in the spectrally multiplexed plane usingthe method of the invention, the rendered composite image 42 typicallyexhibits a confused appearance under conventional ambient lightingconditions; at least one of the source images 12-1, 12-2, etc. is thusdifficult or impossible to distinguish under conventional ambientlighting conditions. A particular source image is made difficult orimpossible to distinguish until a demultiplexer 50 is operated toselectively illuminate the composite image 42 in a manner sufficient toreveal the desired source image. Alternatively, one or more of thesource images may be encoded so as avoid visual confusion and thereforebe visually apparent in the rendered composite image when the renderedcomposite image is subjected to ambient or wide band illumination, andbecome confused or difficult to detect when the rendered composite imageis subjected to a complementary narrow band illuminant.

According to operation of a spectral demultiplexing system 103, aparticular source image (as shown in FIG. 3, source image 12-1) may berecovered and made distinguishable within the composite image 42. In theembodiment illustrated in FIG. 3, the output of the demultiplexer 50 isdirected to an observer 70 using the method of the invention. Therecovered image is then distinguishable by the observer 70 as onesubstantially identical with, or a close approximation of, theparticular source image 12-1 initially provided to the image inputdevice 20.

Recovery of a particular source image will be understood to generallyproceed according to an exemplary embodiment of the spectraldemultiplexing system 103 as follows. The substrate 44 is positionedwith respect to an illuminant source operable within the demultiplexer50, such that a narrow band illuminant generated by the demultiplexer 50illuminates the composite image 42 so as to subject the array ofcolorants in the rendered composite image 42 to the selected illuminant.As a result of the rendered composite image 42 thus being controllablyand selectively illuminated by at least one illuminant, a desired sourceimage is then detectable. In the illustrated embodiment, the desiredsource image is made visually distinguishable to an observer 70. Thedesired source image 12-1, now recovered, is thereby susceptible tocomprehension by an observer 70.

Accordingly, by virtue of the aforementioned interaction of a colorantand its corresponding illuminant, and due to the visual response of theobserver 70 to this particular interaction, each encoded source imagemay be present as a confused, or distinguishable, image depending uponthe objective of the demultiplexing operation.

FIG. 4 is a simplified schematic diagram of exemplary embodiments ofspectral multiplexing, rendering, and spectral demultiplexing methods61, 62, 63, respectively. In step 61 for multiplexing plural sourceimages, a first source image 71 and a second source image 72 areprovided to the multiplexer 30, which outputs a composite image datafile to a rendering device 40. The output of the rendering device 40 issubstrate 90 which has incorporated therein a composite image 92. Theoriginal source image 71 is rendered as a pattern using a firstcolorant; in the illustrated embodiment, a cyan ink or toner is chosen.The second source image 72 is rendered as a pattern using a secondcolorant; in the illustrated embodiment, a magenta ink or toner ischosen. (As there is typically some overlap in absorption bands betweenpractical narrow band colorants, the two source images are preferablyencoded in step 61 to account for the absorption that will occur whenplural colorants are utilized to produce the composite image.)

In a rendering step 62, the composite image specifies patterns in cyanand magenta colorants that are accordingly rendered on a substrate 90 toform the rendered composite image 92. Those skilled in the art willappreciate that certain portions of the two patterns may be co-locatedand other portions are relatively spatially distinct. Nonetheless, incertain embodiments of the present invention, visual recognition of atleast one of the source images in the composite image may be madedifficult or impossible due to the confusion between source images thatare encoded in the composite image.

In step 63 for demultiplexing the rendered composite image 92, thesubstrate 90 having the rendered composite image 92 fixed thereon isilluminated by the demultiplexer 50. Controlled illumination of thesubstrate 90 according to a first mode 51 of illumination causes thefirst source image 71 to achieve a particular level of density withrespect to the remainder of the composite image and thus the firstsource image 71 becomes detectable on the substrate 90. Alternatively,controlled illumination of the substrate 90 according to a second mode52 of illumination causes the second source image 72 to be similarlydetectable on the substrate 90. In the illustrated embodiments, thefirst source image 71 and the second source image 72 are thereforeselectably distinguishable on the substrate 90.

FIG. 5 illustrates a schematic simplified representation of the spectralmultiplexing system 101 of FIG. 3, in which an image processing unit 130and associated peripheral devices and subsystems are employed. An imageinput terminal 120 may include an image capture device 122 such as ascanner, digital camera, or image sensor array; a computer imagegenerator 124 or similar device that converts 2-D data to an image; oran image storage device 126 such as a semiconductor memory or amagnetic, optical, or magneto-optical data storage device. The imageinput terminal 120 derives or delivers digital image data in the formof, for example, plural monochromatic image files, wherein the pictureelements or “pixels” of each image are defined at some gray value. Forexample, the input terminal 120 may be employed to derive an electronicrepresentation of, for example a document or photograph from imagecapture device 122, in a format related to the physical characteristicsof the device, and commonly with pixels defined at m bits per pixel. Ifa color document, the image is defined with two or more separationbitmaps, usually with identical resolution and pixel depth. Image datafrom the input terminal 120 is directed to an image processing unit(IPU) 130 for processing so as to be encoded to create a compositeimage. It will be recognized that the data representing one or moresource images is spectrally encoded by the image processing unit 130 toprovide secondary image data representative of a composite imagesuitable for subsequent rendering.

The image processing unit 130 may include image memory 132 whichreceives input image data from image input terminal 120 or from anothersuitable image data source, such as an appropriately programmed generalpurpose computer (not shown) and stores the input image data in suitabledevices such as random access memory (RAM). Image processing unit 130commonly includes processor 134. The input image data may be processedvia a processor 134 to provide image data representative of pluralsource images defined on respective source image planes in accordancewith the present invention. For example, image data signals in the formof RGB or black and white (B/W) images may be processed, and theluminance information derived therefrom may be used to provide datarepresentative of a source image. Image data signals presented in otherformats are similarly processed: image data signals in, for example theL*a*b format, may be processed to obtain data representing a sourceimage from the lightness channel. Image data signals that are alreadyformatted in grayscale are generally usable without further processing.

Operation of the image processing unit 130 may proceed according to oneor more image processing functions 138, 139 so as to encode the sourceimage data into the composite image file as described hereinabove.Processing may include a color conversion which, if necessary, may beimplemented to convert a three component color description to theprinter-specific four or more component color description, and mayinclude a halftoner which converts a c bit digital image signals to dbit digital image signals, suitable for driving a particular printer,where c and d are integer values. In certain embodiments, additionalfunctions may include one or more of color space transformation, colorcorrection, gamut mapping, and under color removal (UCR)/gray componentreplacement (GCR) functions. Control signals and composite image outputdata are provided to an interface 136 for output from the imageprocessing unit 130.

The image processing unit 130 may be embodied as an embedded processor,or as part of a general purpose computer. It may include special purposehardware such as for accomplishing digital signal processing, or merelyrepresent appropriate programs running on a general purpose computer. Itmay also represent one or more special purpose programs running on aremote computer.

FIG. 6 is a simplified schematic representation of the spectraldemultiplexing system 103 of FIG. 3, in which a controller andassociated peripheral devices and subsystems are employed to present oneor more recovered source images 171, 172. FIG. 6 shows a controller 150connected to a illuminant source 160 that is operable for subjecting thecomposite image 42 on substrate 44 to first and second predefinedilluminants 161, 162. Firstly, as illustrated with reference to therendered composite image 42 on substrate 44, under conventional ambientlighting and in the absence of an illuminant 161, 162, only thecomposite image 42 is distinguishable and no source image is detected.However, upon activation of the source 160 so as to provide the firstpredefined illuminant 161, the recovered source image 171 becomesdetectable to an observer 170. Alternatively, the mode of operation ofthe source 160 may be switched so as to provide a second predefinedilluminant 162, whereupon the composite image 42 is instead subjected tothe second illuminant 162, and the recovered source image 172 becomesdetectable.

In its simplest form the controller 150 may be constructed as amanually-operable illuminant selector switch. Alternatively, asillustrated, the controller 150 may be provided in the form of acomputer-based control device having an interface 156 connected tosource 160, which offers programmable control of the operation of theilluminant source 160. The controller 150 may thus be operated to causeselective activation and deactivation of the illuminant source 160 so asto provide one or more selected fields of illumination 161, 162. Suchcontrol may, for example, the accomplished via manual operation of theilluminant source 160 by a human operator, or by programmable controlafforded by a computer or similar means.

The controller 150 is operable for accomplishing tasks such asactivation, deactivation, or sequencing of the illuminant source 160,setting illuminant intensity, illuminant frequency, etc. Embodiments ofthe controller 150 benefit from operation of a programmable controlsystem comprising standard memory 152 and processor 154. The controller150 may be employed, for example, for supplying uniform R or G or Bscreen images to the interface 156 for subsequent display on theilluminant source 160 when the latter is constructed in the form of aCRT monitor.

Operation of the illuminant source 160 by the controller 150 may proceedaccording to certain sequenced control functions 158, 159 so as toprovide, for example, controlled operation of the illuminant source 160to afford a field of illumination that varies according to selectivecharacteristics such as a sequential activation and deactivation ofselected narrow band illuminants, or of controlled operation of theintensity of same; or with interactive control according to interventionby an operator of the particular sequence, intensity, or duration of theilluminants. As noted above, the rendered composite image may beconstructed to have a plurality of source images encoded therein; forexample, of at least first and second patterns of respective first andsecond colorants. The rendered composite image may be subjected to atemporal sequencing of illumination by respective first and secondnarrow band illuminants, thus allowing a respective one of the first andsecond recovered source images 171, 172 to be sequentiallydistinguishable.

As mentioned, the illuminant source 160 may be provided in the form of aCRT monitor having a screen positionable with respect to the substrate44 for generating the requisite field of illumination sufficient toilluminate the rendered composite image 42. By way of example, thefollowing description gives an example of control settings forgenerating a controlled field of illumination from a CRT monitor underthe control of a desktop computer using a document presentationapplication, such as Microsoft PowerPoint. On a blank slide, a rectangleobject is created that completely covers the extent of the landscapepage. Using the menu selections, the user selects “Format AutoShape”,then “Colors and Lines”. A custom slide color may be specified. Forexample, to create a red slide, set Red to 255, Green to 0, Blue to 0.To create a green slide, set Red to 0, Green to 255, Blue to 0. Tocreate a blue slide, set Red to 0, Green to 0, Blue to 255.

A gray component replacement technique may be applied to the darkness ina recovered image in the areas common to the different colorants whensubjected to their complementary illuminants. For example, whencomposing and rendering a composite image, black can be used to replacea portion of the cyan colorant in the areas of the common darkness thatappear under red light and to the yellow colorant in the areas of thecommon darkness that appear under blue light. Common image darknessproduced with this black component will be more perceptible under whitelight in comparison to the darkness observed under illumination by acyan illuminant or a yellow illuminant. Inclusion of GCR in the encodingand rendering of a composite image will provide a rendered compositeimage having a more confused appearance under white light.

In another aspect of this practice, the GCR fraction employed in thisprocess can be modulated spatially to still further increase or decreasethe desired level of confusion in the composite image.

Furthermore, the GCR fraction employed in this process can be modulatedspatially so as to encode an additional, low resolution source image inthe composite image. When the resulting rendered composite image issubjected to white light illumination, the additional, low-resolutionimage is visually discernible.

Using the cyan/yellow colorant example from above, the white lightillumination problem may be written asd ^(W)(x, y)=d _(C) ^(W)(x, y)+d _(Y) ^(W)(x, y)≈d _(C) ^(W)(x, y)  (3)

Cyan has a much higher density under white light compared to the densityof yellow under white light, so the cyan image may be understood todominate the appearance of a rendered composite image under white light.

Whereas the typical GCR technique uses common density of colorants underthe same illuminant, the contemplated method for implementing GCR in theencoding and rendering of a composite image uses the common density ofcolorants under different illuminants. This common density is consideredherein to the cross-illuminant-common density.

Continuing with the cyan/yellow colorant example, one may select afractional (frac) amount of common density that will be used for black(K) addition and for cyan and yellow (C, Y) subtraction. Assume aprinter linearized in density, the amount of colorant, and the densityof the colorant under the complementary illuminant, in a synonymousfashion. Let:d _(K)(x, y)=frac*min[d _(CR)(x, y), d _(YB)(x, y)]  (4)

This amount of density will be subtracted from d_(CR) to yieldd_(CR-GCR), and from d_(YB) to yield d_(YB-GCR) and the K separationwill be added to the composite image. To first-order, the density of theperceived images are as follows

$\begin{matrix}{{d^{R}\left( {x,y} \right)} = {{{{d_{C - {GCR}}^{R}\left( {x,y} \right)} + {d_{Y - {GCR}}^{R}\left( {x,y} \right)} + {d_{K}\left( {x,y} \right)}} \approx {{d_{C - {GCR}}^{R}\left( {x,y} \right)} + {d_{K}\left( {x,y} \right)}}} = {d_{C}^{R}\left( {x,y} \right)}}} & (5) \\{{d^{B}\left( {x,y} \right)} = {{{{d_{C - {GCR}}^{B}\left( {x,y} \right)} + {d_{Y - {GCR}}^{B}\left( {x,y} \right)} + {d_{K}\left( {x,y} \right)}} \approx {{d_{Y - {GCR}}^{B}\left( {x,y} \right)} + {d_{K}\left( {x,y} \right)}}} = {d_{Y}^{B}\left( {x,y} \right)}}} & (6) \\{{d^{W}\left( {x,y} \right)} = {{{d_{C - {GCR}}^{W}\left( {x,y} \right)} + {d_{Y - {GCR}}^{W}\left( {x,y} \right)} + {d_{K - {GCR}}\left( {x,y} \right)}} \approx {{d_{C - {GCR}}^{W}\left( {x,y} \right)} + {{d_{K}\left( {x,y} \right)}.}}}} & (7)\end{matrix}$

Note that under white light, a fraction of the cross-illuminant-commondensity d_(K), now appears. This additional component yields a whitelight image that appears more confusing than the image described by Eq.(3). The example Illustrated in FIG. 7 is repeated in FIG. 8 with 80%GCR (frac=0.8). FIG. 8 shows that the density under white light differsmore from the red light density image in the GCR image compared to thenon-GCR image illustrated in FIG. 7. In addition to this density effect,the composite image encoded and rendered with GCR has an additional hueeffect that is not illustrated in FIGS. 7 and 8. That is, under whitelight, the regions with different amounts of cyan, magenta, and blackalso exhibit different hues, Thus adding to the confusion.

A composite image encoded in rendered using this GCR method will bediscussed in Examples 1 and 2, below, wherein Example 1 uses cyan andyellow in a manner similar to the above first-order description, andExample 2 uses cyan and magenta.

The aforementioned GCR method may be implemented to encode and render anadditional, low-resolution source image by use of a black (K) colorant.In that case, the fractional GCR component frac is given a spatialdependence according to the additional low-resolution source image.Example 3 discusses such a low-resolution image encoded according to thefraction of GCR that is applied.

A key consideration is that a colorant will absorb some light from anon-complementary illuminant, and thus it will be somewhat discernibleunder that illuminant. To effectively suppress this appearance of aresidual image, one may calibrate the perceived density for eachcolorant and illuminant, and one may encode the source images so as tocompensate for such spurious absorption.

EXAMPLES OF RENDERED COMPOSITE IMAGES GENERATED USING THE GCR TECHNIQUEExample 1 GCR for Image Confusion (as Applied to C/Y Encoded Images)

FIG. 9 is a rendered composite image wherein two source images areencoded in cyan and yellow that are respectively designed for viewingunder red and blue illumination. The image uses a small amount of black(K) to compensate for unwanted absorptions by cyan (C) in the blueregion (so as to make the cyan image less than discernible under blueillumination and to recover the source image in the presence of a yellowilluminant). The use of K greatly increases the dynamic range availablefor encoding the source image. Note that when this rendered compositeimage is viewed under white light, the image in cyan (C) dominates theother confused images and the source image encoded in yellow (Y) ishardly visible.

FIG. 10 is a rendered composite image created with a 80% GCR fraction,wherein the appearance of the rendered composite image under red andblue illumination is substantially identical to the appearance of therendered composite image provided in FIG. 9, but the rendered compositeimage in FIG. 10 under white light is more confused due to theapplication of GCR. One can discern a varying pattern of density and huethat is not immediately apparent as a single image.

Example 2 Use of GCR for Increased Image Confusion

Although the magenta colorant has the highest density under green light(e.g. d_(MG)(255)=1.38), the cyan density under green is high also (e.g.d_(CG)(255)=0.53), which makes it discernible under green light. On theother hand, although the magenta density under blue light is slightlylower than under green light (e.g. d_(MB)(255)=0.98), the cyan densityunder blue is much lower than under green (e.g. d_(CB)(255)=0.24).Accordingly, a rendered composite image that employs a cyan colorant forrepresenting an encoded source image will recover that source imageunder red light; similarly, an encoded source image represented in amagenta colorant is best recover under blue light. A small proportion ofblack colorant may be added to the cyan colorant to make the respectivesource image less discernible under blue light. A small proportion ofblack colorant may be added to the magenta colorant to make therespective source image less discernible under red light. Accordingly,FIGS. 11 and 12 are rendered composite images having encoded thereinfirst and second source images intended for recovery under red and blueilluminants, respectively. FIG. 11 was rendered in cyan and magentacolorants without the application of GCR. FIG. 12 was rendered in cyan,magenta, and black colorants with the application of GCR. In comparingFIGS. 11 and 12 under white light, one may see that rendered compositeimage in FIG. 12 appears more confused under white light in comparisonto the rendered composite image in FIG. 11.

Example 3 Use of Spatially-Varying GCR for Encoding an Additional SourceImage

FIG. 13 is a composite image having encoded therein first and secondsource images intended for recovery under blue and red illuminationwherein GCR has been utilized in the rendering of a composite image incyan and yellow colorants, and a third source image that is encoded inthe composite image for recovery under white light illumination. Theamount of GCR is spatially varied in accordance with the image contentof the third source image. In the rendered composite image of FIG. 13,the image content of the third source image is a binary pattern in theshape of the “digital X” (a trademark of Xerox Corporation), with use ofa 80% GCR fraction in the regions of the composite image that correspondto the image content of the third source image, and no GCR wasimplemented in the remaining regions of the composite image. When theimage is subjected to red or blue light, a respective one of the firstand second source images is recovered. Under white light, the thirdsource image is discernible.

In alternative embodiments, the contemplated third source image mayinclude or be restricted to image content that is encoded for detectionprimarily or exclusively by automated instrumentation (i.e. imagecontent that is designed to be machine-readable rather thanhuman-readable).

In still other embodiments, the third source image may be encoded as agrayscale image by use of a suitable halftoning technique.

Example 4 Varying GCR for Optimized Confusion

FIG. 14 is a rendered composite image that exemplifies an additionalapplication of the contemplated GCR technique, wherein the GCR fractionwas varied randomly between 0 and 80% over square blocks of pixels. Notethat the resulting rendered composite image will reveal the encodedsource images under illumination by red and blue illuminants but imageconfusion is evident in the rendered composite image when subjected towhite light. The level of image confusion may be further optimized bychoosing the image alignments with respect to the particular applicationof the GCR technique. The image confusion may be increased when thefrequency content of the GCR matches that of the dominant image and thedark regions in the two encoded images align so as to allow a selectedamount of variation in GCR.

Note that one skilled in the art may realize a variety of alternativesare within the scope of this invention for implementing the abovedescribed embodiments. Its advantageous use is expected in colorprinting by various processes including offset lithography, letterpress,gravure, xerography, photography, and any other color reproductionprocess which uses a defined number of colorants, usually three or four,in various mixtures. Embodiments of the rendering system 102 includeapparatus capable of depositing or integrating a defined array ofcolorants in a substrate, according to the composite image, such thatthe array of colorants is susceptible to selective reflection ortransmission of a selected narrow band illuminant incident thereon. Forexample, the composite image may be rendered on a transparent film and adesired source image may be recovered when the substrate is backlit by asuitable narrow band illuminant. Examples include hardcopy reprographicdevices such as inkjet, dye sublimation, and xerographic printers,lithographic printing systems, silk-screening systems, and photographicprinting apparatus; systems for imagewise deposition of discretequantities of a color on a substrate surface, such as paint, chemical,and film deposition systems; and systems for integration of colorantmaterials in an exposed surface of a substrate, such as textile printingsystems.

Embodiments of exemplary substrates include, but are not limited to,materials such as paper, cardboard, and other pulp-based and printedpackaging products, glass; plastic; laminated or fibrous compositions;and textiles. Narrow band colorants other than basic CMYK colorants mayalso be used for this invention.

The field of illumination for illuminating a rendered composite imagemay be provided by a variety of illuminant sources that include a narrowband light source responsive to manual control or to program controlaccording to an illuminant source control signal. Various narrow bandlight sources may include apparatus for providing filtered sunlight,filtered incandescent, or filtered fluorescent light; coherent lightsources such as a solid-state laser or laser diode; projection or imagedisplay devices such as those incorporating a cathode ray tube (CRT),flat-panel display (FPD), liquid crystal display (LCD), plasma display,or light emitting diode (LED) and organic light emitting (OLED) arrays.Light sources incorporating a cathode ray tube are advantageous in thatthey have phosphors that exhibit stable and well-understood spectralcharacteristics that are sufficiently narrow and complementary to commonCMY colorants. In addition, such displays are widely available.

Additional familiar components (not shown) may be included such as akeyboard, and a mouse, means for reading data storage media, a speakerfor providing aural cues and other information to the observer, andadapters for connection of the systems described herein to a networkmedium. Computer readable media such as memory, hard disks, CD-ROMs,flash memory, and the like may be used to store a computer programincluding computer code that implements the control sequences pertinentto present invention. Other systems suitable for use with the presentinvention may include additional or fewer subsystems.

Embodiments of the invention are contemplated for providing visualstimulation and amusement, particularly by the inclusion of compositeimages in printed materials such as books or posters, in novelty items,and in software sold to consumers for generating such items. Renderedcomposite images made using this invention can be distributed toconsumers for subsequent demultiplexing when exposed to a field ofillumination generated by, for example, a display device connected to acomputer according to display control signals directed to the computerfrom a remote source, such as from an internet site, or according todisplay control instructions embedded in electronic mail, Internet webpages, or similar transmissions.

Embodiments of the invention may be employed for drawing the attentionof an observer to a particular source of information, such as fordisseminating news, entertainment, or advertising, or to messages orindicia, such as trademarks or product instructions, on objects; tographics, art work, and the like displayed at gathering places suchcinemas, galleries, museums, commercial venues, and trade shows; or tolarge-format displays such as signs, posters, billboards, or murals.Still other embodiments of the invention are contemplated for use inpublications, merchandising, or advertising vehicles such as newspapers,periodicals, or maps; in boxes, bottles, containers, wrappers, labels,or other packaging or shipping materials; in building materialsincluding wall coverings, floor coverings, lighting systems, and thelike.

Other embodiments of the invention are contemplated for implementingspecialized visual effects in a public setting, a performance orentertainment venue, or other gathering place where there is control ofthe ambient lighting. Examples are festivals, theaters, night clubs, andsporting events, where participants may receive printed materials orpackaging, clothing, souvenirs, etc. having incorporated thereon one ormore rendered composite images. Under the influence of localized fieldsof illumination provided by suitably-modified zone lighting equipment,such as stage lighting equipment, which may be synchronized or otherwisecontrolled, a variety of source images having visual interest to theparticipants may be made visible in a dramatic fashion.

Other embodiments of the invention are contemplated for implementingsecure verification of authenticity of a document or other instrument.Such embedded information may be present in the form of a watermark, anindice, or an image useful for validation, secure identification, or thelike. For example, the appearance of a single image or an unchanging(i.e. constant) image viewed under specific, controlled illuminationcould be used to indicate authenticity of a document. Fraudulentattempts to circumvent the verification, such as by proffering aphotocopy or counterfeit of the requisite instrument, may not meet theprinter calibration settings necessary for generating an authenticcomposite image, such that a confused appearance of a composite image ona counterfeit under the controlled lighting would preclude authenticity.Embodiments of the invention are contemplated for providing simpleencryption and decryption of embedded information in documents, coupons,game pieces, tickets, certificates, commercial paper, currency,identification cards, and the like.

Still other embodiments of the invention are contemplated for use intextiles and garments such as head coverings, clothing, and outerwear,and in other wearable or personal items such as footwear, timepieces,eyewear, jewelry, appliques, fashion accessories, and the like. Itemsbearing composite images generated in the course of the practice of thisinvention may have an artistic, novelty, or collectible nature, such asin a souvenir, book, magazine, poster, educational material, tradingcard, or toy.

While particular embodiments have been described, alternatives,modifications, variations, improvements, and substantial equivalentsthat are or may be presently unforeseen may arise to applicants orothers skilled in the art. Accordingly, the appended claims as filed andas they may be amended are intended to embrace all such alternatives,modifications variations, improvements, and substantial equivalents.

1. A method of processing a plurality of source images, comprising thesteps of: encoding the plurality of source images in a composite image,wherein multiple-illuminant GCR is employed to alter to the compositionof one or more source images encoded for rendering as a composite image,and wherein a gray component replacement fraction in themultiple-illuminant GCR is spatially modulated, so as to increaseconfusion; rendering the composite image on a substrate by use of aplurality of colorants; and recovering at least one of the encodedsource images from the rendered composite image, such that the recoveredsource image is made distinguishable, by subjecting the renderedcomposite image to a narrow-band illuminant that is preselected toreveal the source image.
 2. The method of processing a plurality ofsource images of claim 1, wherein a gray component replacement fractionin the multiple-illuminant GCR is implemented to encode an additional,low-resolution source image intended for recovery under illumination bya white light.
 3. A method of processing a plurality of source images toprovide a composite image suitable for rendering as a rendered compositeimage, comprising the steps of: receiving the plurality of sourceimages; and encoding the plurality of source images to thereby providethe composite image; wherein multiple-illuminant GCR is employed toalter the composition of one or more source images encoded for renderingas a composite image by providing a gray component replacement fractionin the multiple-illuminant GCR, so as to provide a selected level ofconfusion between at least two source images, wherein the encodingincludes mapping of source image values at pixel locations in the sourceimages to colorant control values at respective pixel locations in aspectrally-multiplexed image plane, wherein the colorant control valuesspecify an amount of each one of a plurality of M colorants to bedeposited at corresponding locations in the rendered composite image,wherein the mapping of the pixel values from the plurality of sourceimages is determined according to a plurality of spatial luminancedistributions each of which represent the desired response of therendered composite image to illumination thereof by a respective one ofa plurality of N illuminants, and wherein the mapping of the pixelvalues from the plurality of source images is calculated to cause aselected one of the source images to be recovered when the renderedcomposite image is subject to illumination by at least one of the Nilluminants.
 4. The method of claim 3, wherein the gray componentreplacement fraction is spatially modulated.
 5. The method of claim 3,where the gray component replacement fraction is spatially modulated soas to encode at least one of the source images in the composite image.6. The method of claim 3, wherein the mapping of pixel values includesadjusting the colorant control values to compensate for unwantedabsorption of at least one of the N illuminants by at least one of thecolorants.
 7. The method of claim 3, wherein at least two of thecolorants are selected from the group consisting of cyan, magenta,yellow, and black colorants, and at least two of the N illuminants arenarrow-band illuminants selected from the group consisting of red,green, and blue illuminants.
 8. The method of claim 3, wherein themapping of pixel values includes adjusting the colorant control valuesto produce first and second spatial luminance distributions when therendered composite image is subjected to respective first and secondnarrow-band illuminants, the first spatial luminance distribution havinga constant density, and the second spatial luminance distribution havinga spatially-varying density.
 9. The method of claim 3, wherein themapping of pixel values includes a gamut mapping step to limit themapping to a predetermined system gamut according to a determination ofrealizable luminance values.
 10. The method of claim 3, furthercomprising the step of rendering the composite image to produce therendered composite image on a substrate.
 11. An article of manufacturecomprising a substrate having rendered thereon a rendered compositeimage produced according to the method of claim
 10. 12. An imagingsystem for receiving image data representative of plural source imagesand for processing the image data to thereby provide a composite imagein a form suitable for rendering as a rendered composite image,comprising: an image processing unit for receiving the plurality ofsource images and for encoding the plurality of source images to therebyprovide the composite image; wherein multiple-illuminant GCR is employedto alter the composition of one or more source images encoded forrendering as the rendered composite image by providing a gray componentreplacement fraction in the multiple-illuminant GCR, so as to provide aselected level of confusion between at least two source images; whereinthe encoding includes mapping of source image values at pixel locationsin the source images to colorant control values at respective pixellocations in a spectrally-multiplexed image plane; wherein the colorantcontrol values specify an amount of each one of a plurality M ofcolorants to be deposited at corresponding locations in the renderedcomposite image; wherein the mapping of the pixel values from theplurality of source images is determined according to a plurality ofspatial luminance distributions each representing the desired responseof the rendered composite image to illumination thereof by a respectiveone of a plurality of N illuminants, and the mapping of the pixel valuesis calculated to cause a selected one of the source images to berecovered when the rendered composite image is subject to illuminationby at least a selected one of the N illuminants; and an interface forproviding the composite image.
 13. The imaging system of claim 12,further comprising: an image rendering device for receiving thecomposite image and for rendering the composite image on a substrate toprovide the rendered composite image; and a demultiplexer for subjectingthe rendered composite image to illumination by the selected illuminantso as to recover the selected source image.
 14. A computer programembodied on a computer readable medium, the program being executable forreceiving image data representative of plural source images and forprocessing the image data to thereby provide a composite image suitablefor rendering as a rendered composite image, comprising the steps of:receiving the plurality of source images; and encoding the plurality ofsource images to thereby provide the composite image; whereinmultiple-illuminant GCR is employed to alter the composition of one ormore source images encoded for rendering as a composite image byproviding a gray component replacement fraction in themultiple-illuminant GCR, so as to provide a selected level of confusionbetween at least two source images, wherein the encoding includesmapping of source image values at pixel locations in the source imagesto colorant control values at respective pixel locations in aspectrally-multiplexed image plane, wherein the colorant control valuesspecify an amount of each one of a plurality of M colorants to bedeposited at corresponding locations in the rendered composite image,wherein the mapping of the pixel values from the plurality of sourceimages is determined according to a plurality of spatial luminancedistributions each of which represent the desired response of therendered composite image to illumination thereof by a respective one ofa plurality of N illuminants, and wherein the mapping of the pixelvalues from the plurality of source images is calculated to cause aselected source image to be recovered when the rendered composite imageis subject to illumination by at least one of the N illuminants.
 15. Theprogram of claim 14, wherein the gray component replacement fraction isspatially modulated.
 16. The program of claim 14, where the graycomponent replacement fraction is spatially modulated so as to encode atleast one of the source images in the composite image.
 17. The programof claim 14, wherein the mapping of pixel values includes adjusting thecolorant control values to compensate for unwanted absorption of atleast one of the N illuminants by at least one of the colorants.
 18. Theprogram of claim 14, wherein at least two of the colorants are selectedfrom the group consisting of cyan, magenta, yellow, and black colorants,and at least two of the N illuminants are narrow band illuminantsselected from the group consisting of red, green, and blue illuminants.19. The program of claim 14, wherein the mapping of pixel valuesincludes adjusting the colorant control values to produce first andsecond spatial luminance distributions when the rendered composite imageis subjected to respective first and second narrow band illuminants, thefirst spatial luminance distribution having a constant density, and thesecond spatial luminance distribution having a spatially-varyingdensity.
 20. The program of claim 14, wherein the mapping of pixelvalues includes a gamut mapping step to limit the mapping to apredetermined system gamut according to a determination of realizableluminance values.
 21. The program of claim 14, further comprising thestep of rendering the composite image to produce the rendered compositeimage on a substrate.
 22. The program of claim 14, wherein the Nilluminants include a first illuminant having a wide band powerdistribution and a second illuminant having a narrow band powerdistribution.
 23. The program of claim 14, wherein the M colorantsinclude cyan, magenta, yellow, and black colorants, and the second modeof illumination employs a narrow band illuminant selected from the groupconsisting of red, green, and blue illuminants.
 24. The program of claim14, wherein the mapping of pixel values includes adjusting the colorantcontrol values to compensate for unwanted absorption by at least one ofthe M colorants.